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Chapter  1 


Wide  Field  Integration 

1.1  Review  of  Optic  Flow  Formulation 

Standard  closed  loop  control  design  techniqnes  involve  sensing,  estimation,  and 
actnation.  Typically,  the  sensing  and  control  procedures  are  fast  procedures.  How¬ 
ever,  the  estimation  procedures  generally  induce  the  most  latency  into  the  system. 
This  slows  how  quickly  a  closed  loop  system  can  respond  to  environmental  factors. 
Biological  systems  do  not  use  the  massive  amounts  of  processing  and  estimation  for 
navigation  and  control  that  their  human  created  mechanical  counterparts  do.  In¬ 
stead,  insects  in  particular  use  a  more  efficient  method  for  closed  loop  control.  They 
eliminate  the  estimation  process  completely  and  directly  feed  the  sensory  input  to 
their  actuation  systems.  This  form  of  Wide  Field  Integration  (WFI)  takes  the  vi¬ 
sual  information  in  tangential  cells  in  their  eyes  and  compares  it  to  predetermined 
patterns.  The  differences  in  the  patterns,  being  picked  up  by  their  eyes  and  the 
predetermined  patterns  are  then  used  to  actuate  the  insect  in  order  to  avoid  obsta¬ 
cles.  Using  multiple  patterns  to  compare  against  the  perceived  optic  flow  allows  for 
parallel  processing,  which  further  decreases  an  insects  response  time  to  stimuli. 
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Figure  1.1:  Planar  tunnel  geometry.  (A)  Notation  and  vehicle  state  definitions  (B) 
Balanced  planar  nearness  function  and  lateral/rotational  perturbations  of  /r. 


Definitions  for  Planar  Optic  Flow  Qix,  r,  7) 


Q  =  -uj - ( i;,  [-  sin  7  0057]^  ) 

r 


Figure  1.2:  Environment  and  Optic  Flow  Generation 
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1.1.1  Summary  of  Planar  Optic  Flow 


Optic  flow,  a  relative  measure  of  speed/depth,  provides  a  relative  proximity 
and  speed  with  respect  to  obstacles  in  the  environment.  This  can  be  used  for  navi¬ 
gation  and  collision  avoidance.  Figure  1.2  shows  a  vehicle  in  a  general  environment 
and  the  apparent  velocity  of  the  objects  in  its  field  of  view.  The  apparent  velocity  is 
perpendicular  to  the  ray  from  the  center  of  the  vehicle  to  the  edge  of  the  object  in 
question.  The  distance  r(7,  q),  where  q  is  the  position  and  orientation  of  the  vehicle 
and  7  is  the  viewing  angle,  is  bounded  between  0  and  cx).  If  the  minimum  value  of 
r(7,  q)  is  forced  to  be  some  small,  but  non-zero  number  e  then  the  reciprocal. 


,q)  =  /  ^  X 

r(7,q) 

is  bounded  between  0  and  some  large,  but  hnite  number.  This  allows  the  formulation 
of  an  optic  flow  function  Q,  which  is  piecewise  continuous,  and  might  have  a  hnite 
number  of  discontinuities.  For  a  2-D  planar  case,  the  equation  is  given  as 


Q(7,q,q)  =  -0  +  /i(7,q)  (ifeSinq-T/ftCosq),  (1.2) 

where  —9  is  the  rotational  component,  and  /i(7,q)  (if,  sin 7  —  yhcosj)  is  the  trans¬ 
lational  component,  which  contains  information  about  the  distances  to  objects  in 
the  environment. 
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Wighting  Function  Tunnel  Geometry  Linearization  (xq) 


37r(a2-y2)  sin  0  COS  9 

cos  27  -  2(^2^  Xb  cos  6^ 

Table  1.1:  Spatial  Fourier  Decomposition  of  Planar  Tunnel  Optic  Flow  for  Vehicles  with 
Sideslip  Constraint  (Reproduced  from  [4]) 

1.1.2  WFI  Sensory  Outputs 

Previous  work  [2]  and  [3]  shows  that  pertnrbations  form  a  desired  pattern 
of  optic  flow  can  be  characterized  by  a  small  number  of  Fourier  harmonics,  which 
correlate  to  relative  proximity  and  speed  with  respect  to  the  environment.  The  in¬ 
formation  extraction  model,  termed  Wide-Field  Integration,  is  given  in  equation  1.3 
where  Ui  is  the  outpnt  corresponding  to  a  particnlar  weighting  function  Fj. 


dro  Q 

S-Tra 


7/ 

2a2  y 


1  ■ 

Ui  =  {Q,Fi{-f))  =  -  g(7,q,q)Fi(7)  dy  (1.3) 

^  Jo 

1.2  WFI  Implementation  on  the  Vehicle 

The  practical  application  of  WFI  to  the  vehicle  involves  feeding  back  the  hrst 
and  second  cosine  harmonics  for  navigation.  Table  I  shows  the  linearized  corre¬ 
spondence  from  states  of  the  vehicle  to  Fonrier  coefficients.  Equation  1.4  shows  the 
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general  control  scheme  for  the  vehicle  where  u  is  the  rotational  velocity  input  and 


is  used  to  actuate  a  centering  response. 


u  =  Ki  {Q,  cos  7)  +  K2  {Q,  cos  27) 


(1.4) 
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Chapter  2 


Vehicle  Design  and  Specifications 
2.1  Overview 

The  specifications  for  the  vehicle  are  given  as  a  ground  vehicle  which  can  carry 
the  electronics  necessary  to  compute  optic  flow,  have  a  top  speed  of  1  m/s,  can  be 
no  larger  than  1  ft  in  diameter,  and  must  be  able  to  rotate  in  place.  A  secondary 
specihcation  is  that  the  vehicle  could  drive  both  forwards  and  backwards.  Also, 
the  vehicle  needs  to  be  kinematic,  that  is,  well  modeled  by  hrst  order  differential 
equations  rather  than  the  second  order  equations  derived  from  physics. 

The  hrst  version  of  the  vehicle  was  constructed  from  scratch.  The  chassis  is 
designed  to  be  two  disks,  mounted  one  above  another.  The  motors,  wheels,  and 
computer  are  designed  to  be  mounted  on  the  lower  disk,  while  the  cameras,  mirrors, 
and  other  various  pieces  of  equipment  are  mounted  on  the  upper  disk.  The  motors 
are  designed  to  be  controlled  by  PIC  microprocessors,  and  contain  a  feedback  loop 
using  wheel  encoders  for  sensors.  However,  the  encoders  used  were  too  coarse  to 
maintain  hne  control  in  the  lower  speed  regime.  This  leads  to  the  acquisition  of 
a  prepackaged  system  to  supply  the  chassis  and  drive  system.  Figures  2.1  and  2.2 
show  the  original  prototype. 

The  acquisition  of  the  prepackaged  chassis  leads  to  the  relaxing  of  the  diameter 
of  the  vehicle,  which  is  now  15  in.  However,  the  secondary  specihcations  are  met 
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Figure  2.1:  Original  Robot  Design  1 


with  this  system  as  well  as  the  slightly  relaxed  primary  specihcations.  While  the 
current  vehicle  does  not  meet  the  letter  of  the  initial  specihcations,  it  meets  the 
spirit  of  them. 

2.2  Vehicle  Hardware 

The  vehicle  used  is  build  off  of  an  X80  robot  chassis  made  by  Dr.  Robot.  The 
X80s  motor  control  board,  drive  motors,  battery,  and  wheel  encoders  are  all  off  the 
shelf  items  from  Dr.  Robot.  The  base  X80  system  is  show  in  hgure  2.4.  All  the  Dr. 
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Figure  2.2:  Original  Robot  Design  2 


Robot  systems  are  run  by  their  proprietary  software,  taking  in  position  or  velocity 
commands  for  the  wheels,  and  returning  recorded  velocities  and  positions  of  each 
wheel.  The  connection  to  the  low  level  controller  on  the  vehicle  is  a  serial  connection. 
The  main  onboard  computer  is  essentially  a  miniaturized  desktop  computer  built  off 
of  a  Biostar  motherboard  with  a  2.4  GHz  AMD  CPU.  The  hard  drive  is  a  250  GB 
7200  RPM  drive  produced  by  Sea  Gate.  A  hre  wire  PCI  expansion  board  by  Point 
Grey  Research  provides  the  interface  between  the  computer  and  the  onboard  camera. 


There  are  two  main  sources  for  power  for  the  vehicle:  an  8.4  V  NiMH  battery 


Figure  2.3:  Information  and  Actuation  Flows 


supplied  by  Dr.  Robot  which  runs  the  X80  power  systems,  and  a  15.8  V,  10,000  amp- 
hour  LiPo  which  supplies  the  onboard  computer.  These  batteries  provide  enough 
power  to  operate  the  vehicle  for  at  least  two  hours  of  continuous  use.  Depending 
upon  the  load  placed  on  the  electrical  system,  this  can  be  extended  considerably. 
However,  there  is  no  good  model  for  power  consumption  beyond  the  two  hour  use. 
Communication  with  the  computer  is  done  via  wireless.  The  vehicle  is  typically 
run  using  a  remote  desktop  protocol,  though  direct  interfacing  through  a  monitor, 
keyboard  and  mouse  is  also  possible.  Figure  2.3  shows  how  the  different  components 
on  the  vehicle  interact  with  each  other  and  the  environment,  and  hgure  2.5  shows 
the  hnal  vehicle  conhguration. 
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Figure  2.4:  X80  Vehicle 


2.3  Dynamic  Model  and  System  ID 

In  order  to  talk  meaningfully  about  control  a  dynamic  model  of  the  vehicle 
must  be  determined.  It  is  assumed  that  vehicle  motion  can  be  described  with  the 
following  kinematic  model 
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Figure  2.5:  Final  Vehicle  Configuration 


X  =  Xb  cos  9 

y  =  XbsinO  (2.1) 

6  =  Ui 

(2.2) 

where  it  is  assumed  that  Xb  =  Vq  =  constant.  To  test  the  validity  of  these  as¬ 
sumptions,  a  system  identification  test  was  performed  on  the  vehicle.  The  following 
frequency  domain  model  was  assumed: 
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l"(s)  =  Gt(s)Xfe(s) 


(2.3) 


U{s)  =  Gr{s)e{s) 

where  Gt{s)  =  and  Gr{s)  =  and  V,U  were  the  translational  and 

rotational  inputs  respectively.  If  the  kinematic  assumption  is  valid,  Kt  —  Kr  —  1 
and  Tt  ^Tr  ^  0. 

Data  on  the  vehicle’s  position  and  velocity  is  collected  using  the  Vicon  System. 
This  system  is  capable  of  retro  reflective  tracking  markers  on  a  vehicle  at  350  Hz. 
The  system  makes  a  model  of  the  vehicle  and  returns  positions  and  orientations. 
The  number  of  markers  and  placement  dictates  the  error  in  the  data.  However, 
most  models  give  an  accuracy  of  less  than  2  mm,  and  an  orientation  of  less  than 
0.1  radians.  Several  markers  are  placed  asymmetrically  on  the  vehicle  for  tracking. 
A  model  is  created  in  the  Vicon  software.  The  vehicle  stores  all  commands  sent 
to  the  wheels,  in  addition  to  external  sensors  and  time  stamps  of  the  data.  To 
control  the  vehicle,  a  Labview  program  was  created  that  takes  in  commands  from  a 
joystick  and  translates  them  to  wheel  velocity  commands.  Two  different  modes  are 
modeled  during  the  tests:  translation  and  rotation.  Using  the  joystick,  a  frequency 
rich  series  of  commands  is  sent  to  the  vehicle.  To  synchronize  the  data  between 
the  Vicon  system  and  the  vehicle  data,  an  integrated  rate  gyro  is  placed  on  the 
vehicle.  This  is  used  with  the  heading  information  provided  by  Vicon  to  match  the 
vehicle  data  with  the  positions  and  velocities.  At  the  start  of  each  test,  a  series  of 
pseudo-random  rotations  are  made  to  make  this  process  easier  [1].  The  data  on  both 
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Vicon  and  on  the  onboard  computer  is  taken  at  500  Hz.  Due  to  issues  with  memory 
allocation  during  the  start  of  a  program,  the  onboard  computer  data  is  linearly 
interpolated  to  match  the  Vicon  data.  This,  with  the  initial  rotations,  allows  the 
data  from  the  two  systems  to  be  correlated. 

The  results  show  that  a  hrst  order  model  predicts  the  behavior  of  the  vehicle 
well,  both  in  rotation  and  in  translation. 

Results  for  the  models  and  the  recorded  data  can  be  seen  on  Figure  2.6  for 
translation,  and  Figure  2.8  for  rotation.  Figures  2.7  and  2.9  show  that  the  model 
tracks  the  general  behavior,  but  is  unable  to  account  for  the  higher  order  oscillations 
in  the  velocity. 

Kt  is  determined  to  be  1.140,  and  Tt  is  0.0376.  Likewise,  Kr  is  1.066  and 
is  0.0209.  A  second  order  mode  is  also  present  with  a  natural  frequency  of  about 
14  Hz,  but  is  much  smaller  in  magnitude  than  the  hrst  order  mode.  Since  the  time 
constants  Tt  and  are  very  small,  the  kinematic  approximation  is  a  good  model 
for  the  vehicle. 

2.4  Vehicle  Closed  Loop  Performance 

In  order  to  verify  the  vehicle  performance,  several  different  tests  are  run.  An 
important  test  is  the  vehicle’s  closed  loop  performance.  One  variant  of  the  vehicle 
uses  a  single  Firehy  camera  looking  up  at  a  parabolic  mirror  to  obtain  a  full  held  of 
view  of  the  environment  around  the  vehicle.  Using  the  optic  how  algorithms  devel¬ 
oped  previously  and  in  [4],  the  vehicle  is  capable  of  traversing  a  number  corridors 
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Velocity  vs.  Time 


Figure  2.6:  Measured  and  modeled  outputs  for  the  Translational  system 


and  cluttered  environments  without  colliding  with  obstacles.  Figure  2.10  shows  the 
performance  of  the  vehicle  in  a  corridor  with  a  90°  bend,  as  well  as  the  Fourier 
coefficients  and  the  optic  ffow  at  various  points.  In  this  experiment,  the  vehicle 
was  run  closed  loop  down  the  corridor  with  just  the  sensory  information  from  the 
onboard  camera.  The  vehicle  was  started  near  and  pointed  towards  the  right  wall. 
Figure  2.  IOC  shows  the  optic  flow  for  the  vehicle  turning  to  avoid  the  wall,  as  well 
as  the  large  mismatch  in  the  magnitude  of  the  left  and  right  sides  optic  flow.  Once 
the  vehicle  has  centered  itself  in  the  corridor,  it  then  turns  right  to  avoid  crashing 
into  the  walls  and  attempts  to  stay  centered  on  its  new  heading.  Figure  2.10D  shows 
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Velocity  vs.  Time 


Figure  2.7:  Measured  and  modeled  outputs  for  the  Translational  system 


a  typical  optic  flow  pattern  for  driving  down  the  middle  of  a  straight  corridor  or 
hallway. 
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Velocity  vs.  Time 


Figure  2.8:  Measured  and  modeled  outputs  for  the  Rotational  system 
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Figure  2.9:  Measured  and  modeled  outputs  for  the  Rotational  system 
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Figure  2.10:  Vehicle  performance,  Fourier  coefficients,  and  optic  flow. 

(A)  shows  the  closed  loop  performance  donw  a  corridor  with  a  90°  bend, 

(B)  shows  the  Fourier  coefficients  during  the  experiment,  and  (C)  and 
(D)  show  the  optic  flow  at  two  different  points  during  the  experiment. 


Chapter  3 


Experimental  Setup 

3.1  Camera  Configuration 

Two  different  cameras  are  used  for  these  experiments.  These  are  a  Thermal- 
Eye  4550AS  camera  from  L-3  communications  and  a  Firefly  MV  camera  from  Point 
Grey  Research.  The  field  of  view  on  the  Thermal-Eye  camera  is  50°  and  the  field 
of  view  of  the  Firefly  is  30°.  Both  cameras  are  mounted  facing  the  same  direction, 
perpendicular  to  the  forward  direction  of  the  vehicle.  The  held  of  view  for  the 
Thermal-Eye  is  given  in  the  data  sheet  shipped  with  the  camera.  The  held  of  view 
for  the  Firehy  is  measured,  since  only  part  of  the  Firehy’s  maximum  image  capture 
size  is  used. 

A  Matrox  Morphis  Quad  frame  grabber  card  is  used  to  collect  the  images 
from  the  Thermal-Eye  camera.  Due  to  hardware  conhicts  between  the  chipset  on 
the  robot  and  the  frame  grabber  card,  the  Thermal-eye  images  are  taken  and  stored 
on  an  oh-board  computer.  While  the  Thermal-eye  is  mounted  on  the  vehicle,  the 
power  and  communications  with  it  are  run  on  a  tether  to  a  desktop  computer.  All 
the  data  is  stored  on  the  desktop. 

The  Firehy  camera  is  plugged  directly  into  the  PCI  hre  wire  card  on  the 
vehicle.  All  data  and  power  for  this  camera  is  supplied  by  the  vehicle  itself  and  all 
data  is  stored  on  the  onboard  computer.  The  frame  rate  of  the  Firehy  in  closed  loop 
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is  30  fps  +/-  .1  fps. 


3.2  Physical  Layout 

Two  lights,  each  with  three  40  W  light  bulbs,  are  placed  approximately  2  ft 
away  from  the  vehicle.  These  lights  are  controlled  by  a  dimmer  switch  which  allows 
the  full  range  of  light  settings  from  darkness  to  the  maximum  light  output  for  the 
lamps.  These  lights  are  placed  parallel  to  the  wall,  either  in  line  with  the  vehicle  for 
the  rotation  experiments,  or  1  ft  behind  the  vehicles  travel  path  for  the  translation 
experiments.  The  texture  used  for  the  experiments  is  a  thermal  blanket  draped  over 
the  walls  of  the  environment.  The  coils  in  the  blanket  are  set  to  increase  the  heat 
by  5°.  The  actual  temperature  difference  is  irrelevant,  however,  as  long  as  there 
is  enough  thermal  contrast  between  the  coils  in  the  blanket  and  the  environment. 
The  visual  texture  of  the  thermal  blanket  is  sufficient  to  generate  image  shifts. 
Figures  3.1  and  3.2  show  the  physical  layout  for  the  rotational  experiment. 

3.3  Light  Measurement 

A  Sekonic  L-359  Light  Master  light  meter  is  used  to  take  all  light  value  mea¬ 
surements.  This  light  meter  is  typically  used  by  photographers  to  establish  how  fast 
the  shutter  speed  on  a  camera  should  be  set  at  and/or  how  long  and  how  bright  a 
flash  should  be  set.  While  the  light  meter  has  many  different  types  of  data,  only  the 
Exposure  Value  (EV)  readings  are  used.  The  other  output  options  are  more  camera 
specihc.  The  conversion  between  EV  and  lumens  or  candle  light  per  square  foot  is 
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Figure  3.1:  Rotation  Experiment  Setup 
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Figure  3.2:  Rotation  Experiment  Setup 
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given  in  equation  3.1.  The  low  EV  values  for  the  various  experiments  (  0-0.5  EV) 
are  at  about  the  light  level  of  a  dark  parking  lot  at  night.  The  high  EV  values  (  4-5 
EV)  corresponds  to  lighting  high  enough  to  see  clearly,  but  not  overly  bright. 


EV  =  log2{ 


lumens 

10 


+  10 


(3.1) 


3.4  Experiment  Procedures 

3.4.1  Pure  Rotation 

The  pure  rotation  experiments  have  the  vehicle  turning  at  a  constant  0.25 
rad/sec  turn  rate  for  a  set  duration  of  time.  The  average  distance  from  the  robot 
to  the  textures  being  viewed  is  6  ft  +/-  8  in.  The  light  measurements  are  taken  at 
three  different  points  along  the  arc  of  the  circle,  one  at  either  end  of  the  arc,  and 
one  at  the  center.  The  arc  that  the  cameras  on  the  vehicle  view  is  8  ft  +/-  1ft,  or 
about  1.3  radians. 


3.4.2  Pure  Translation 

The  translation  experiments  have  the  vehicle  moving  in  a  straight  line  parallel 
to  a  wall.  The  distance  from  the  wall  to  the  cameras  is  3.5  ft  +/-  3  in.  The  speed  of 
the  vehicle  was  8  in/s  +/-  1  in/s.  Light  measurements  are  taken  at  three  different 
points  along  the  wall:  at  the  beginning,  middle,  and  end. 
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3.5  Image  Processing 


3.5.1  Method  of  Processing  Data 

Data  from  the  experiments  is  received  as  a  series  of  images.  Additional  in¬ 
formation,  namely  frame  rates  and  vehicle  ideal  optic  flow,  are  also  supplied.  To 
process  the  information,  an  appropriate  sequence  of  images  is  chosen.  This  is  done 
by  throwing  out  the  images  that  correspond  either  to  when  the  vehicle  not  moving, 
or  during  the  initial  jerk  of  starting  and  stopping.  The  remaining  images  are  then 
run  through  a  processing  program  to  determine  the  image  shift  between  frames. 
Each  pair  of  frames  (n,  n  -f  1)  is  taken  in  by  the  program,  then  40  lines  of  pixel  shift 
are  computed  along  horizontal  lines.  These  lines  are  averaged  vertically,  giving  a 
single  line  of  average  pixel  shifts  across  the  entire  image.  Next,  the  lines  of  optic 
flow  are  averaged  to  give  a  single  average  pixel  shift  for  the  entire  image.  Finally, 
the  average  pixel  shift  per  image  is  averaged  across  the  entire  data  set,  giving  the 
hnal  global  average  of  pixel  shift  for  the  experiment.  Using  the  frame  rate  and  the 
physical  parameters  from  the  experiment,  that  is  the  held  of  view  of  the  camera,  the 
pixel  shift  is  converted  to  units  of  rad/s  and  compared  to  the  ideal  optic  how.  The 
conversion  from  pixels  in  the  image  to  held  of  view  is  made  by  dividing  the  number 
of  horizontal  pixels  by  the  held  of  view  measured  from  a  static  image.  Equation  3.2 
gives  the  conversion  from  the  pixels  per  frame  to  radians  per  second. 

pixel  frame  FOV  rad 

Qdot  =  7  ^  7  (3-2) 

frame  sec  pixel  o 
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3.6  Methods  of  Determining  Image  Shift 


3.6.1  Gradient  Method 

The  gradient  method  for  determining  image  shifts  is  merely  solving  equa¬ 
tion  3.3  along  a  set  line  on  the  image.  ^  is  evaluated  as  the  estimated  image  shift 
between  two  frames  in  time.  ^  is  the  shift  in  intensities  along  the  prescribed  line. 
One  problem  with  this  method  is  that  when  only  using  data  along  the  prescribed 
lines,  the  ability  to  track  pixels,  which  get  shifted  off  of  the  given  line  is  lost.  This 
gives  a  tendency  to  give  false  readings,  particularly  if  there  is  any  vibration  in  the 
image. 


dx  dx  dl 
dt  dl  dt 


(3.3) 


3.6.2  Lucas-Kanade 

The  Lucas-Kanade  method  is  built  off  of  the  gradient  method  [5].  One  of  the 
major  improvements  from  the  gradient  method  is  the  fact  that  this  works  in  two 
dimensions,  effectively  using  the  entire  image  rather  than  just  a  prescribed  line. 
This  method  uses  a  form  of  non-linear  optimization  to  converge  on  an  image  shift. 
In  order  to  do  this,  a  low  resolution  version  of  the  image  is  processed  for  an  initial 
estimate  on  image  shift.  Once  an  estimate  is  found,  a  slightly  higher  resolution 
is  used  from  the  original  image,  with  the  estimated  initial  image  shift  used  as  the 
starting  point  for  the  optimization  routine.  This  continues  for  a  prescribed  number 
of  iterations.  The  result  is  a  two  dimensional  vector  held  for  the  shift  of  each  pixel 
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in  the  image.  The  benehts  of  this  method  over  the  gradient  method  are  that  all  of 
the  data  from  the  image  can  be  used,  rather  than  just  data  along  a  line.  However, 
the  improved  estimates  come  at  the  expense  of  increased  computation  time. 

3.6.3  Data  Reduction 

Once  the  image  shift  per  pixel  on  a  line  is  been  determined,  the  average  pixel 
shift  per  line  is  computed.  These  lines  are  then  averaged  again,  resulting  in  a 
global  average  across  the  picture.  This  is  valid,  since  the  experiments  consist  of 
either  constant  translation  or  rotation.  There  should  be  no  local  image  changes 
such  as  expansion  or  contraction.  Finally,  the  average  shifts  between  images  are 
averaged  over  the  duration  of  the  experiment,  giving  a  global  average  shift  across 
the  experiment.  Given  the  physical  dimensions  of  the  experiment  and  the  velocity 
of  the  vehicle,  the  ideal  optic  flow  is  calculated  and  compared  against  the  computed 
optic  flow. 

In  addition  to  the  average  optic  flow  over  time,  the  data  is  given  temporally 
as  well.  At  each  data  point  the  data  is  show  as  a  histogram.  These  histograms  are 
generated  by  taking  the  optic  flow  between  each  pair  of  images  and  binning  it  for 
each  test.  The  Firefly  camera  generates  a  data  point  every  0.033  sec  of  test  time 
(with  images  being  taken  at  30  Hz),  and  the  Thermal- Eye  camera  every  0.091  sec 
(a  frame  rate  of  11  Hz).  The  final  value  in  the  average  plots  is  just  the  average  of  all 
the  data  shown  in  each  histogram.  Typically,  a  few  of  the  data  points  in  the  high 
end  of  the  histogram  come  from  the  initial  start  of  the  experiment,  and  a  few  of  low 
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end  points  are  influenced  by  the  inertia  of  the  system  even  though  the  motors  have 
been  told  to  stop. 
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Chapter  4 


Experimental  Results 

4.1  Low  Light  Conditions 

The  low  light  experiments  measure  the  amount  of  optical  flow  detected  by  the 
infrared  and  visible  light  cameras  under  various  lighting  conditions.  These  range 
from  near  darkness  to  standard  indoor  lighting.  The  results  show  a  near  constant 
estimate  of  optic  flow  from  the  Thermal-Eye  camera  over  the  entire  range  of  testing. 
However,  the  Firefly  loses  the  ability  to  track  objects  as  the  light  levels  drop. 

4.1.1  Translation 

The  translational  data  (Figures  4.1,  4.5  and  4.3)  shows  that  the  optic  flow 
detected  by  the  Thermal-Eye  camera  is  constant  over  the  entire  light  range,  while 
the  Firefly  camera’s  ability  to  detect  shifts  in  images  drops  off.  This  is  expected 
since  the  Firefly  camera’s  ability  to  detect  image  shifts  disappears  as  the  images 
approach  a  uniform  black  image. 

4.1.2  Rotations 

The  results  from  the  rotation  experiments  are  similar  to  the  translational  ex¬ 
periments.  Notably,  the  quality  of  the  visible  light  optic  flow  falls  off  as  the  environ¬ 
ment  gets  darker,  while  the  infrared  optic  flow  quality  is  unaffected.  In  figures  ?? 
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Figure  4.1:  Measured  Optic  Flow  from  the  Low  Light  Translational  Experiments 


and  4.6  it  can  be  seen  that  the  data  spred  is  less  in  some  of  the  data  points  corn- 
paired  to  others.  This  is  because  the  data  came  from  two  different  sets  of  tests. 
This  variation  comes  from  differences  in  the  charge  of  the  battery  which  runs  the 
wheels  and  the  low  level  control. 

4.2  Obscured  Line  of  Sight 

To  obscure  the  line  of  sight  between  the  vehicle  and  its  environment,  a  liquid 
based  fog  is  pumped  into  the  air.  The  lighting  is  the  same  in  the  clear  environment 
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Figure  4.2:  Temporal  Data  from  the  Low  Light  Infrared  Translational 
Experiments 


as  in  the  bright  end  of  the  low  light  condition  tests.  To  quantify  the  level  of  fog,  a 
black  and  white  image,  figures  4.7  and  4.8,  is  posted  within  the  image  field  for  the 
Firefly  camera.  For  each  test,  the  intensity  of  the  black  in  the  image  is  compared 
to  the  intensity  of  the  white  in  the  image  as  show  in  equation  4.1.  In  order  to  get 
consistent  results,  the  light  and  dark  areas  are  the  same.  In  the  case  where  the 
black  and  white  image  is  completely  obscured,  an  estimate  is  made  using  the  same 
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Figure  4.3:  Temporal  Data  from  the  Low  Light  Visible  Translational  Experiments 


method  and  location  of  where  the  image  should  be  based  upon  the  initial  conditions 
of  the  test.  To  get  the  different  levels  of  fogginess,  fog  is  pumped  into  the  test  area. 
The  tests  are  then  run,  as  the  fog  slowly  dissipates  out  of  the  test  area.  In  order  to 
keep  the  fog  from  dissipating  too  quickly,  plastic  sheets  are  used  to  enclose  the  test 
section. 
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Optic  flow  vs.  Light  Level 


Figure  4.4:  Measured  Optic  Flow  from  the  Low  Light  Rotational  Experiments 
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4.2.1  Translation 

The  data  from  the  translational  tests  (hgure  4.9)  shows  that  signihcant  optic 
flow  is  detected  at  all  ranges  of  fog.  This  is  due  to  the  fact  that  the  fog  is  not 
perfectly  uniform  and  thus  the  shift  of  the  image  can  be  detected.  This  however 
raises  a  problem,  since  the  Firefly  camera  is  no  longer  giving  information  about 
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Figure  4.5:  Temporal  Data  from  the  Low  Light  Infrared  Rotational  Experiments 


the  environment  being  traversed,  but  instead  giving  information  about  the  local  air 
velocity  in  the  local  vicinity.  Figures  4.10  and  4.11  show  the  data  arrayed  temporally. 

4.2.2  Rotation 

Like  the  fog  translation  experiments,  the  rotation  case  (hgure  4.12)  also  shows 
the  same  issues  with  optic  flow  being  detected,  but  not  relevant  to  the  task  of 
navigation  and  control.  Like  the  translation  experiment,  the  rotational  data  is  also 
shown  temporally  in  hgures  4.13  and  4.14. 
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Temporal  Data  from  the  Low  Light  Visible  Rotational  Experiments 
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Figure  4.7:  Calibration  Images  from  Fog  Translational  Tests 
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Figure  4.8:  Calibration  Images  from  Fog  Rotation  Tests 
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Figure  4.9:  Measured  Optic  Flow  from  the  Fog  Translational  Experiments 
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Figure  4.10:  Temporal  Data  from  the  Visible  Light  Translational  Fog  Experiments 
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Figure  4.11:  Temporal  Data  from  the  Infrared  Translational  Fog  Experiments 


39 


Optic  flow  (rad/sec) 


Optic  flow  vs.  Visiblity 
0.4 - ! - 1 - 1 - 

0.35  - 


0.3 

0.25 

0.2 

0.15 

0.1 


A 


d: 


A 


-1^ 


'  I 


- E3 - EJ 


L\ 


0.05 


0 

0  20  40  60  80  100 


— B —  Idea 
-X-  Visible  Light 
-A  -  Infrared 


%  Visiblity 


Figure  4.12:  Measured  Optic  Flow  from  the  Fog  Rotational  Experiments 
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Figure  4.13: 


Temporal  Data  from  the  Visible  Light  Rotational  Fog  Experiments 
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Figure  4.14:  Temporal  Data  from  the  Infrared  Rotational  Fog  Experiments 
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Chapter  5 


Conclusions 

The  vehicle  presented  in  this  thesis  meets  a  modihed  set  of  the  initial  parame¬ 
ters.  This  vehicle  is  capable  of  testing,  among  other  things,  the  viability  of  infrared 
optic  flow  for  navigation  and  control.  Two  sets  of  experiments  document  this;  a  set 
of  low  light  experiments,  and  a  set  of  reduced  visibility  experiments.  The  low  light 
experiments  show  that  as  the  light  level  drops  the  ability  for  a  visible  light  sensor 
to  compute  meaningful  optic  flow  decreases.  This  trend  is  the  same  in  both  the 
translational  and  rotational  experiments.  This  is  expected,  since  in  total  darkness, 
there  would  be  no  change  in  the  image  to  generate  optic  flow.  Unlike  the  visible  light 
spectrum,  infrared  light  is  unaffected  by  a  loss  in  visible  light  directly.  The  only 
way  that  this  could  change  is  if  the  light  source  produces  enough  infrared  radiation, 
and  there  is  little  thermal  contrast  naturally  in  the  environment. 

The  obscured  vision  tests  show  that  low  levels  of  fog  does  not  seem  to  affect 
optic  flow.  However  once  the  visibility  drops  to  between  10  to  20%  the  optic  flow 
drops  off.  The  actual  usefulness  of  this  optic  flow  can  be  somewhat  questionable, 
since  the  relative  motion  of  the  fog  to  the  vehicle  can  change  the  measured  optic 
flow.  Like  the  low  light  experiments,  the  rotation  and  translation  experiments  show 
the  same  trends.  Also,  the  infrared  optic  flow  is  unaffected  but  the  fog,  which  again 
is  expected. 
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There  are  several  avenues  of  research  based  off  of  these  experiments.  One 
branch  is  to  quantify  the  how  obscured  visibility  effects  the  closed  loop  performance 
of  the  vehicle.  Open  loop  tests  show  that  there  is  signihcant  optic  flow  under  high 
levels  of  fog.  It  is,  however,  undetermined  if  this  optic  flow  is  useful  for  navigation 
and  control.  Closed  loop  control  experiments  are  one  of  the  ways  to  determine  what 
level  of  vision  obstruction  causes  control  problems.  Another  research  direction  is  to 
develop  methods  for  using  infrared  and  normal  cameras  in  unison. 
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Appendix  A 
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Figure  A.l:  Data  from  the  Low  Light  Translational  Experiments 
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Figure  A. 2:  Data  from  the  Low  Light  Rotational  Experiments 
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Figure  A. 3:  Data  from  the  Fog  Translational  Experiments 
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Figure  A. 4:  Data  from  the  Fog  Rotational  Experiments 
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